Biometric authentication device, biometric authentication method, and recording medium

ABSTRACT

A biometric authentication device is provided with: a replay unit for reproducing a sound; an ear authentication unit for acquiring a reverberation sound of the sound in an ear of a user to be authenticated, extracting an ear acoustic feature from the reverberation sound, and calculating an ear authentication score by comparing the extracted ear acoustic feature with an ear acoustic feature stored in advance; a voice authentication unit for extracting a talker feature from a voice of the user that has been input, and calculating a voice authentication score by comparing the extracted talker feature with a talker feature stored in advance; and an authentication integration unit for outputting an authentication integration result calculated based on the ear authentication score and the voice authentication score. After the sound is output into the ear, a recording unit inputs the voice of the user.

This application is a Continuation of U.S. application Ser. No.17/270,303 filed on Feb. 22, 2021, which is a National Stage Entry ofPCT/JP2019/032729 filed on Aug. 22, 2019, which claims priority fromJapanese Patent Application 2018-162229 filed on Aug. 31, 2018, thecontents of all of which are incorporated herein by reference, in theirentirety.

TECHNICAL FIELD

The example embodiments relate to a biometric authentication device andthe like for authenticating a user.

BACKGROUND ART

A personal authentication technology using biometric information on auser, such as a fingerprint, iris, face, voiceprint, and ear, has theadvantage of being less likely to be leaked or stolen and not to beforgotten or lost than a password or key. For this reason, variousbiometric authentication approaches for authenticating users have beendeveloped in recent years, including voiceprint (voice) authentication(hereinafter referred to as voice authentication) and optoacousticauthentication (hereinafter referred to as ear authentication).

In voice authentication, the user's voice is used for authentication.For example, in an instrument such as a smart speaker, a user performsan operation through voice interaction with an artificial intelligence(AI) agent equipped in the instrument. Voice authentication is used forpurposes such as logging in to a service provided by the instrument orproviding a more suitable response to an individual user from theinstrument (personalizing) (see PTL 1).

In ear authentication, reverberation characteristics in an ear of a userare used for authentication. For example, a hearable device such asheadphones or earphones is equipped with a microphone, and extracts thereverberation characteristics of a sound in the ear to authenticate auser. The microphone of the hearable device can also be used tosimultaneously perform the above voice authentication (see PTL 2).

CITATION LIST Patent Literature

-   [PTL 1] JP 2002-221990 A-   [PTL 2] JP 2015-206857 A

SUMMARY OF INVENTION Technical Problem

When the above voice authentication and ear authentication are performedindependently, the authentication is not precise in some cases. Forexample, when the background noise is large, the accuracy of the voiceauthentication is lowered, and when the earphone is not sufficientlyadhered to the ear, the accuracy of the ear authentication is lowered.If authentication fails, the user cannot immediately use an instrument(or function) accessible on the assumption that authentication succeeds.This could bring about fatal consequences for users who need to use theinstrument immediately (for example, emergency medical personnel).Meanwhile, when voice authentication and ear authentication are simplycombined, although the accuracy becomes higher, the time taken byauthentication is given as the sum of times required for theseapproaches of authentication, and the user has to wait for a longertime.

The disclosure has been made in view of the above disadvantages, and oneobject of the disclosure is to provide a biometric authentication deviceand the like capable of performing highly accurate, quick authenticationin combination of voice authentication and ear authentication.

Solution to Problem

In view of the above disadvantages, a biometric authentication deviceaccording to a first aspect of the disclosure includes:

-   -   a replay means reproducing a sound;    -   an ear authentication means acquiring a reverberation sound of        the sound output from the replay means in an ear of a user to be        authenticated, extracting an ear acoustic feature from the        reverberation sound, and calculating an ear authentication score        by comparing the extracted ear acoustic feature with an ear        acoustic feature stored in advance;    -   a voice authentication means extracting a talker feature from a        voice of the user that has been input, and calculating a voice        authentication score by comparing the extracted talker feature        with a talker feature stored in advance; and    -   an authentication integration means outputting an authentication        integration result calculated based on the ear authentication        score and the voice authentication score, wherein    -   after the sound is output into the ear by the replay means, the        voice of the user is input.

A biometric authentication method according to a second aspect of thedisclosure includes:

-   -   acquiring a reverberation sound of a sound output from a replay        means in an ear of a user to be authenticated, extracting an ear        acoustic feature from the reverberation sound, and calculating        an ear authentication score by comparing the extracted ear        acoustic feature with an ear acoustic feature stored in advance;    -   extracting a talker feature from a voice of the user that has        been input, and calculating a voice authentication score by        comparing the extracted talker feature with a talker feature        stored in advance; and    -   outputting an authentication integration result calculated based        on the ear authentication score and the voice authentication        score, wherein    -   after the sound is output into the ear by the replay means, the        voice of the user is input.

A biometric authentication program according to a third aspect of thedisclosure includes:

-   -   acquiring a reverberation sound of a sound output from a replay        means in an ear of a user to be authenticated, extracting an ear        acoustic feature from the reverberation sound, and calculating        an ear authentication score by comparing the extracted ear        acoustic feature with an ear acoustic feature stored in advance;    -   extracting a talker feature from a voice of the user that has        been input, and calculating a voice authentication score by        comparing the extracted talker feature with a talker feature        stored in advance; and    -   outputting an authentication integration result calculated based        on the ear authentication score and the voice authentication        score,    -   the biometric authentication program causing a computer to        enable that    -   after the sound is output into the ear by the replay means, the        voice of the user is input.

The program may be stored on a non-transitory computer-readable storagemedium.

Advantageous Effects of Invention

According to the disclosure, a biometric authentication device and thelike capable of performing highly accurate, quick authentication incombination of voice authentication and ear authentication can beprovided.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a configuration diagram of a biometric authentication deviceaccording to a first example embodiment of the disclosure.

FIG. 2 is a configuration diagram of an earphone.

FIG. 3 is a sequence chart illustrating the operation of the biometricauthentication device according to the first example embodiment of thedisclosure.

FIG. 4 is a configuration diagram of a biometric authentication deviceaccording to a second example embodiment of the disclosure.

FIG. 5 is a sequence chart illustrating the operation of the biometricauthentication device according to the second example embodiment of thedisclosure.

FIG. 6 is a configuration diagram of a biometric authentication deviceaccording to a third example embodiment of the disclosure.

FIG. 7 is a configuration diagram of an information processing deviceusable in the first to third example embodiments.

EXAMPLE EMBODIMENT

Hereinafter, several example embodiments will be described in detailwith reference to the drawings.

First Example Embodiment

(Biometric Authentication Device)

A biometric authentication device 100 according to a first exampleembodiment will be described with reference to FIG. 1 . The biometricauthentication device 100 is equipped with an electronic circuitincorporating a voice authentication process, an ear authenticationprocess, and a process of integrating results of both of the processes.Alternatively, a program for executing these processes may beimplemented by being installed in a communication instrument (such as asmartphone) owned by a user. As illustrated in FIG. 1 , the biometricauthentication device 100 is connected to an earphone 1 built with aspeaker 2 and a microphone 3 via a wired cable or wirelesscommunication. The speaker 2 is connected to a replay unit 111 of thebiometric authentication device 100. The microphone 3 is connected to arecording unit 112 of the biometric authentication device 100.

As illustrated in FIG. 2 , the biometric authentication device 100 maybe equipped on the earphone 1 built with the speaker 2 and themicrophone 3.

The speaker 2 outputs a sound to a user side. In ear authentication, themicrophone 3 inputs a reverberation sound from an ear side of the userwhile the user is wearing the earphone 1. In voice authentication, themicrophone 3 inputs a speaking of the user while the user is wearing theearphone 1. When it is difficult to pick up the user's voice duringvoice authentication, the user may move the earphone 1 near the mouth orthe like. The microphones 3 may be prepared separately for voiceauthentication and ear authentication. For example, using an earphone ona headset side, a microphone for ear authentication may be equippedinside the earphone 1, and a microphone alone such as an earphonemicrophone may be fixed near the mouth of the user as a microphone forvoice authentication.

The communication instrument (not illustrated) owned by the user may beequipped with an AI interaction program. The AI interaction program canbe operated in cooperation with various applications, and is capable ofacquiring various types of information and executing various applicationfunctions through interaction using voice. The voice on the AI side isdelivered to the user via the speaker 2 built in the earphone 1, and thevoice on the user side is delivered to the AI interaction program viathe microphone 3 built in the earphone 1. The above-mentionedinformation acquisition and function execution includes information andfunctions that are only permitted to be accessed by an authorized user.Examples of the above include functions that require strong security toprotect personal information, such as connecting to an email account anda bank account. When requested by the user to access such a function,the AI interaction program activates the biometric authentication device100 to manage the access.

As illustrated in FIG. 1 , the biometric authentication device 100according to the first example embodiment includes an interactioncontrol unit 101, the replay unit 111, the recording unit 112, an earauthentication unit 121, a voice authentication unit 131, and anauthentication integration unit 141.

The interaction control unit 101 controls in such a way that, after thereplay unit 111 outputs a sound into the user's ear, the recording unit112 inputs (records) the user's voice. The interaction control unit 101includes an ear authentication notification unit 102, a voiceauthentication notification unit 103, an ear authentication instructionunit 104, a voice authentication instruction unit 105, and anauthentication result storage unit 106.

The ear authentication notification unit 102 notifies the user that earauthentication (or both of ear authentication and voice authentication)will begin. Specifically, the ear authentication notification unit 102causes the replay unit 111 to reproduce a voice (which can be a recordedvoice) for letting the user know that the authentication is to start.

The voice authentication notification unit 103 notifies the user thatvoice authentication will begin. Specifically, the voice authenticationnotification unit 103 causes the replay unit 111 to reproduce a voice(which can be a recorded voice) for letting the user know thatauthentication is to start.

The ear authentication instruction unit 104 instructs the earauthentication unit 121 to start ear authentication.

The voice authentication instruction unit 105 instructs the voiceauthentication unit 131 to start voice authentication. The voiceauthentication instruction unit 105 or the voice authentication unit 131may generate a one-time password that can be used only for apredetermined time. The generated one-time password is delivered to theuser by voice via the earphone 1, or is delivered to the user by beingdisplayed on a display or the like of the user's communicationinstrument (not illustrated).

The authentication result storage unit 106 stores verification scoresnotified from the ear authentication unit 121 and the voiceauthentication unit 131.

The replay unit 111 reproduces (outputs) a recorded sound. The replayunit 111 may include a storage unit for storing a sound to bereproduced.

The recording unit 112 records a sound that is input. The recording unit112 may include a storage unit for storing a recorded sound.

The ear authentication unit 121 includes a signal generation unit 122,an acoustic characteristic extraction unit 123, an ear verification unit124, and a feature storage unit 125.

The signal generation unit 122 generates a sound (such as a probesignal) to be output from the replay unit 111. This sound is output toacquire a reverberation sound in the ear of the user to beauthenticated.

The acoustic characteristic extraction unit 123 acquires thereverberation sound in the ear of the user to be authenticated, via therecording unit 112, and extracts acoustic characteristics relevant tothe ear shape of the user from the acquired reverberation sound.

The ear verification unit 124 verifies whether acoustic characteristicsof the user to be authenticated, which are stored in advance in thefeature storage unit 125, coincide with the extracted acousticcharacteristics. The result of the verification is output as averification score (ear authentication score).

The feature storage unit 125 stores the acoustic characteristics of theuser to be authenticated, which have been acquired in advance.

The voice authentication unit 131 includes a talker feature extractionunit 132, a voice verification unit 133, and a feature storage unit 134.

The talker feature extraction unit 132 extracts a talker feature fromthe voice of the user recorded via the recording unit 112.

The voice verification unit 133 verifies whether a talker feature of theuser to be authenticated, which is stored in advance in the featurestorage unit 134, coincides with the extracted talker feature. Theresult of the verification is output as a verification score (voiceauthentication score).

The feature storage unit 134 stores the talker feature of the user to beauthenticated, which has been acquired in advance.

The authentication integration unit 141 calculates and outputs anauthentication integration result, based on the ear authentication scoreand the voice authentication score.

The authentication integration result to be output is passed to aprogram that manages access to the secured instrument or function, suchas the aforementioned AI interaction program. If the authenticationintegration result is equal to or more than a predetermined value, theAI interaction program permits access to a function desired by the userand terminates the biometric authentication device 100.

(Operation of Biometric Authentication Device)

The operation of an authentication process of the biometricauthentication device 100 according to the first example embodiment willbe described below with reference to the sequence chart illustrated inFIG. 3 .

First, in step S1, when the interaction control unit 101 receives thefact that biometric authentication is required due to a request or thelike from the user, the ear authentication notification unit 102 of theinteraction control unit 101 instructs the replay unit 111 to reproducea voice for notifying the user of the start of ear authentication andvoice authentication. The replay unit 111 reproduces a voice (forexample, “ear authentication is starting and then voice authenticationwill follow”) for notifying the user of the start of ear authenticationand voice authentication, in line with the instruction. The notificationmay be made not only by voice but also on a screen. For example, whenthe user owns a communication instrument (not illustrated) provided witha display unit, “authentication is starting” may be displayed on theprovided display unit.

In step S2, the ear authentication instruction unit 104 instructs theear authentication unit 121 to start ear authentication. Upon receivingthis instruction, the signal generation unit 122 of the earauthentication unit 121 generates a probe signal and sends the generatedprobe signal to the replay unit 111.

In step S3, the replay unit 111 transmits the probe signal to thespeaker 2, and the speaker 2 reproduces the transmitted probe signal.The probe signal may be an audible sound such as a beep sound or amelody, or may be a white noise. The probe signal may be a non-audiblesound such as ultrasound.

In step S4, the recording unit 112 records a reverberation sound of thereproduced probe signal in the user's ear via the microphone 3, andsends the recorded data to the ear authentication unit 121. The earverification unit 124 of the ear authentication unit 121 performs an earauthentication process using the recorded data. In the earauthentication process, the ear verification unit 124 estimates atransfer function in the ear using the probe signal and thereverberation sound. The ear verification unit 124 may further compressthe transfer function into a low-dimensional feature by principalcomponent analysis or the like, or may convert the transfer functioninto a feature such as a mel-frequency cepstral coefficient (MFCC). Theoutput thus obtained is employed as an ear acoustic feature. The earverification unit 124 compares the obtained ear acoustic feature withanother ear acoustic feature exactly of the concerned user extracted inadvance by the same procedure at the time of user registration, andcalculates an ear acoustic score. A Euclidean distance or a cosinedistance is conceivable as a distance scale used for score calculation.

In step S5, the ear verification unit 124 sends the verification scoreand the like as results of the ear authentication process to theinteraction control unit 101. The interaction control unit 101temporarily stores the received verification score of the earauthentication in the authentication result storage unit 106.

In step S6, the voice authentication notification unit 103 of theinteraction control unit 101 instructs the replay unit 111 to reproducea voice for notifying the user of the start of voice authentication anda prompt for the voice authentication. At this time, since the user hasalready recognized in step S1 that the authentication process is beingexecuted, it is unnecessary to make a notification directly indicatingthe start of execution of voice authentication, such as “voiceauthentication is starting”, but it is sufficient to give only aspecific prompt such as “please say your password”. The replay unit 111reproduces a voice for notifying the user of the start of voiceauthentication in line with the instruction.

As the password, a matter that is normally known only to the user (forexample, “please say your mother's maiden name”) may be used. Words usedin voice authentication may be notified by voice (for example, “pleasesay today's date”). Besides, the password may be displayed on thedisplay unit (not illustrated) of the user's communication instrument insuch a way that the user is notified of the password (for example,“please say the password: ABC” is displayed). The voice authenticationinstruction unit 105 may generate a one-time password every time apredetermined time elapses, and cause the replay unit 111 to broadcastthe generated password and deliver the generated password to the uservia the earphone 1 by sound, or cause the display unit (not illustrated)of the user's communication instrument to display the generatedpassword. With this method, the degree of safety of the password can beenhanced.

In step S7, the voice authentication instruction unit 105 of theinteraction control unit 101 instructs the voice authentication unit 131to start voice authentication. Upon receiving the instruction, the voiceauthentication unit 131 instructs the recording unit 112 to startrecording the user's speaking (password).

In step S8, the recording unit 112 records the voice of the user who hasheard the voice authentication start notification and started anutterance via the microphone 3, and sends the recorded data to the voiceauthentication unit 131.

In step S9, the voice authentication unit 131 performs the voiceauthentication process using the received recorded data. The voiceauthentication unit 131 extracts the talker feature such as an i-vectorfrom the recorded voice uttered by the user. The talker feature may beextracted using a neural network capable of identifying the talker.Furthermore, the extracted talker feature is compared with anothertalker feature exactly of the concerned user extracted in advance by thesame procedure at the time of user registration, and the verificationscore is calculated. A cosine distance, probabilistic lineardiscriminant analysis (PLDA), and the like are used for scorecalculation. Prior to the calculation of these scores, preprocesses suchas average normalization, whitening, and norm normalization may beperformed. After the voice authentication process, the voiceauthentication unit 131 sends the voice authentication results such asthe verification score to the interaction control unit 101.

In step S10, once the interaction control unit 101 receives two types ofauthentication results (verification scores) for ear authentication andvoice authentication, the interaction control unit 101 sends theseauthentication results to the authentication integration unit 141.

In step S11, the authentication integration unit 141 integrates both ofthe authentication results, and finally examines whether the user beingauthenticated is exactly the concerned user or another person. For theintegration of the verification scores, any score integration approachsuch as a score average or a weighted average may be used. Theauthentication integration unit 141 determines whether the calculatedauthentication integration score exceeds a preset threshold value, anddetermines that the authentication is successful when the calculatedauthentication integration score exceeds the threshold value, whiledetermining that the authentication is failed when the calculatedauthentication integration score does not exceed the threshold value.When notified from the user or another constituent member that there isan abnormality or defect in either the ear authentication or the voiceauthentication, the interaction control unit 101 may instruct theauthentication integration unit 141 to weight a verification score forwhich the authentication has been successfully completed. Thedetermination result is transmitted to the interaction control unit 101as the authentication integration result. When the authenticationintegration result indicates the successful authentication, theinteraction control unit 101 notifies the AI interaction program or thelike that manages access that the user is permitted to acquire theinformation or execute the function as requested. When theauthentication integration result indicates the authentication failure,the above AI interaction program or the like is notified to that effect.At this time, the interaction control unit 101 may notify the user ofthe authentication integration result via the earphone 1 or the like.

As described above, the operation of the authentication process of thebiometric authentication device 100 is ended.

In the example embodiment, the description has been given assuming aprocess (program) that operates in the communication instrumentpossessed by the user, but this is not limited to the process in theportable communication instrument. For example, the program for theauthentication process may be implemented in a server, and thecommunication instrument of the user may receive only the authenticationintegration result from the server via a network. Alternatively, theearphone 1 may be directly equipped with an instrument or componentcapable of executing the program.

Effects of First Example Embodiment

The biometric authentication device 100 according to the exampleembodiment can perform authentication with higher accuracy incombination of voice authentication and ear authentication. This isbecause the authentication integration unit 141 calculates and outputsthe authentication integration result based on the ear authenticationscore output from the ear authentication unit 121 and the voiceauthentication score output from the voice authentication unit 131. Whenvoice authentication and ear authentication are performed independently,the accuracy of authentication is lowered due to a variety ofcircumstances. However, the biometric authentication device 100 executesvoice authentication and ear authentication using the biometricinformation on the user acquired via the earphone 1, and uses theauthentication integration result of the voice authentication and theear authentication, thereby being able to authenticate the user withoutlowering the accuracy. Moreover, the biometric authentication device 100integrates the verification results of both of the voice authenticationand the ear authentication, thereby being able to obtain a higherauthentication accuracy as compared with the case of performing eitherauthentication.

In the biometric authentication device 100, the voice authenticationprocess is executed after the ear authentication process, that is, atleast, the voice of the user is input by the recording unit 112 after asound is output into the ear by the replay unit 111. With thisprocedure, the authentication process can be completed earlier as awhole than in a normal device in which the ear authentication processand the voice authentication process are simply combined, which in turncan make the waiting time of the user shorter.

Second Example Embodiment

In the first example embodiment, the user is notified by voice twice,namely, at the start of ear authentication and at the start of voiceauthentication. In the example embodiment, an approach for shorteningthe overall time required for authentication by reducing the number oftimes of notification to the user will be described.

(Biometric Authentication Device)

As illustrated in FIG. 4 , a biometric authentication device 200according to the example embodiment includes an interaction control unit201, a replay unit 111, a recording unit 112, an ear authentication unit121, a voice authentication unit 131, and an authentication integrationunit 141.

The interaction control unit 201 includes a simultaneous notificationunit 202, a voice authentication notification unit 103, an earauthentication instruction unit 104, and a voice authenticationinstruction unit 105. The simultaneous notification unit 202 instructsthe replay unit 111 to reproduce a prompt for letting a user perform earauthentication and voice authentication consecutively, by voice.

The other units and devices are the same as in the first exampleembodiment.

(Operation of Biometric Authentication Device)

An authentication operation of the biometric authentication device 200of the example embodiment will be described with reference to thesequence chart illustrated in FIG. 5 .

First, in step T1, when the interaction control unit 201 receives thefact that biometric authentication is required due to a request or thelike from the user, the simultaneous notification unit 202 of theinteraction control unit 201 instructs the replay unit 111 to reproduce,by voice, a prompt for letting the user perform ear authentication andvoice authentication consecutively, such as “please say your passwordafter the beep sound”. This is to start voice authenticationconsecutively, swiftly after the generation of a probe sound for earauthentication. Although the probe signal may be a non-audibleultrasound or the like in the first example embodiment, an audible soundis used in the example embodiment. This is because, unless the audiblesound is used, the user cannot know at what timing to begin uttering thepassword (to start voice authentication), and furthermore, the audiblesound has higher accuracy in the acoustic characteristics of thereverberation sound. The replay unit 111 reproduces a voice fornotifying the user of the start of ear authentication and voiceauthentication via a speaker 2 in accordance with the instruction.

In step T2, the ear authentication instruction unit 104 of theinteraction control unit 201 instructs the ear authentication unit 121to start ear authentication.

In step T3, a signal generation unit 122 of the ear authentication unit121 generates a probe signal and sends the generated probe signal to thereplay unit 111. The replay unit 111 reproduces the sent probe signaland outputs the reproduced probe signal to the speaker 2 of an earphone1.

In step T4, the recording unit 112 records a reverberation sound of thereproduced probe signal in the user's ear via the microphone 3, andsends the recorded data to an ear verification unit 124.

In step T5, the ear verification unit 124 performs ear authenticationusing the recorded data, and sends an ear authentication result such asthe verification score to the interaction control unit 201.

In step T6, once the ear authentication is completed, the voiceauthentication instruction unit 105 of the interaction control unit 201instructs the voice authentication unit 131 to start voiceauthentication.

In step T7, the voice authentication unit 131 instructs the recordingunit 112 to start recording the user's voice. The recording unit 112records the utterance of the user who has started the utterance, inaccordance with the instruction, and sends the recorded data to thevoice authentication unit 131.

In step T8, the voice authentication unit 131 performs voiceauthentication using the recorded data, and sends a voice authenticationresult such as the verification score to the interaction control unit201.

In step T9, upon receiving the authentication result (verificationscore) for voice authentication, the interaction control unit 201 sendsthe authentication results for ear authentication and voiceauthentication to the authentication integration unit 141.

In step T10, the authentication integration unit 141 integrates theauthentication results for ear authentication and voice authentication,and outputs an authentication integration result of finally examiningwhether the user being authenticated is exactly the concerned user oranother person, to the interaction control unit 201.

Effects of Second Example Embodiment

In the example embodiment, in addition to the effects of the firstexample embodiment, the overall time required for authentication can beshortened. This is because the simultaneous notification unit 202 of theinteraction control unit 201 notifies the user in such a way that earauthentication and voice authentication can be performed consecutively,in other words, it is not necessary to make a notification of the startof voice authentication after the ear authentication is completed.Therefore, ear authentication and voice authentication can be performedconsecutively with one prompt to the user, and the time taken to obtainthe authentication integration result can be shortened.

Modification 1 of Second Example Embodiment

In the second example embodiment described above, the verification scorecomputation for ear authentication (step T5) is followed by the voicerecording for voice authentication (steps T6 and T7), but theverification score computation for ear authentication and the voicerecording for voice authentication may be executed in parallel. Inspecific description, the ear verification unit 124 may send areverberation sound recording completion notification to the interactioncontrol unit 201 before ear authentication is performed, and theinteraction control unit 201 may disclose step T6 with the reverberationsound recording completion notification as a trigger, instead of theverification score for ear authentication.

With this procedure, ear authentication and voice authentication can beperformed consecutively with one prompt to the user, and the time takento obtain the authentication integration result can be further shortenedas compared with the second example embodiment.

Modification 2 of Second Example Embodiment

By informing the user in advance that “the probe signal is a sign tostart voice authentication”, it is also possible to eliminate theprocess in step T1 illustrated in FIG. 5 (the simultaneous notificationunit 202 reproduces a prompt for causing ear authentication and voiceauthentication to be performed consecutively, for the replay unit 111 byvoice). That is, step T1 is omitted, and steps T2 to T10 are performedin the order of execution. The user may be informed by display on thedisplay unit (not illustrated) of the communication instrument used bythe user, or may be informed by hand or by a paper medium at the startof the biometric authentication device 200.

With this configuration, as compared with the second example embodimentand modification 1 of the second example embodiment, the time taken toobtain the authentication integration result can be further shortened byan amount corresponding to omitting step T1.

Modification 3 of Second Example Embodiment

Instead of step T6 (the voice authentication instruction unit 105 of theinteraction control unit 201 instructs the voice authentication unit 131to start voice authentication) illustrated in FIG. 5 , the voiceauthentication instruction unit 105 may instructs the voiceauthentication unit 131 to start voice authentication after waiting forthe length of continuation of the probe signal, at the same time as stepT2, or immediately before or after step T2. This is because, since theprobe signal has a fixed length of continuation, the waiting time isalso a fixed time, and the timing of starting voice authentication canbe measured. At this time, the verification score computation for earauthentication and the voice recording for voice authentication may beexecuted in parallel without waiting for the verification scorecomputation result for ear authentication.

With this configuration, as compared with the second example embodimentand modification 1 of the second example embodiment, the time taken toobtain the authentication integration result can be further shortened byan amount corresponding to omitting step T6.

Third Example Embodiment

A biometric authentication device 300 according to a third exampleembodiment will be described with reference to FIG. 6 . The biometricauthentication device 300 is provided with: a replay unit 11 thatreproduces a sound; a recording unit 12 that records a sound; an earauthentication unit 13 that extracts an ear acoustic feature from areverberation sound of the sound output from the replay unit in an earof a user to be authenticated, after the reverberation sound is recordedby the recording unit, and calculates an ear authentication score bycomparing the extracted ear acoustic feature with an ear acousticfeature stored in advance; a voice authentication unit 14 that extractsa talker feature from a voice of the user that has been recorded via therecording unit, and calculates a voice authentication score by comparingthe extracted talker feature with a talker feature stored in advance;and an authentication integration unit 15 that outputs an authenticationintegration result calculated based on the ear authentication score andthe voice authentication score, in which, after the sound is output intothe ear by the replay unit 11, the recording unit 12 inputs the voice ofthe user.

According to the third example embodiment, authentication with higheraccuracy can be performed quickly in combination of voice authenticationand ear authentication. This is because the authentication integrationunit 15 calculates and outputs the authentication integration result,based on the ear authentication score and the voice authenticationscore. Furthermore, in the biometric authentication device 300, thevoice authentication process is executed after the ear authenticationprocess, that is, at least, the voice of the user is input by therecording unit 12 after a sound is output into the ear by the replayunit 11. With this procedure, the authentication process can becompleted earlier as a whole than in a normal device in which the earauthentication process and the voice authentication process are simplycombined.

(Information Processing Device)

In each of the above-described example embodiments, some or all of theconstituent elements of the biometric authentication devices illustratedin FIGS. 1, 4, 6 , and other drawings can be achieved using anycombination of an information processing device 500 and a program asillustrated in FIG. 7 , for example. The information processing device500 includes, as an example, the following constituents.

-   -   Central processing unit (CPU) 501    -   Read only memory (ROM) 502    -   Random access memory (RAM) 503    -   Storage device 505 that stores program 504 and other data    -   Drive device 507 that performs reading and writing on recording        medium 506    -   Communication interface 508 connected to communication network        509    -   Input/output interface 510 that inputs and outputs data    -   Bus 511 that connects between respective constituent elements

Each constituent element of the biometric authentication device in eachexample embodiment of the application is achieved by the CPU 501acquiring and executing the program 504 that enables the functions ofthese constituent elements. The program 504 that enables the function ofeach constituent element of the biometric authentication device isstored in advance, for example, in the storage device 505 or the RAM503, and is read out by the CPU 501 as necessary. The program 504 may besupplied to the CPU 501 via the communication network 509, or may bestored in advance in the recording medium 506 in such a way as to besupplied to the CPU 501 by the drive device 507 that has read out thestored program.

The method of achieving each device has various modifications. Forexample, the biometric authentication device may be achieved by anycombinations of information processing devices and programs that areseparate from each other for each constituent element. A plurality ofconstituent elements included in the biometric authentication device maybe achieved by any combination of one information processing device 500and one program.

Some or all of multiple constituent elements of the biometricauthentication device are achieved by other general-purpose or dedicatedcircuitry, a processor, or the like, or a combination thereof. Theseconstituent elements may be constituted by a single chip or a pluralityof chips connected via a bus.

Some or all of multiple constituent elements of the biometricauthentication device may be achieved by a combination of the circuitryor the like described above and a program.

In a case where some or all of multiple constituent elements of thebiometric authentication device are achieved by a plurality ofinformation processing devices, pieces of circuitry, or the like, theplurality of information processing devices, pieces of circuitry, or thelike may be centrally arranged or dispersedly arranged. For example, theinformation processing devices, pieces of circuitry, or the like may beachieved as a form in which the devices or pieces of circuitry or thelike are connected with each other via a communication network, such asa client and server system or a cloud computing system.

While the invention has been particularly shown and described withreference to exemplary embodiments thereof, the invention is not limitedto these embodiments. It will be understood by those of ordinary skillin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the example embodimentsas defined by the claims.

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2018-162229, filed on Aug. 31, 2018, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   -   1 earphone    -   2 speaker    -   3 microphone    -   100 biometric authentication device    -   101 interaction control unit    -   102 ear authentication notification unit    -   103 voice authentication notification unit    -   104 ear authentication instruction unit    -   105 voice authentication instruction unit    -   106 authentication result storage unit    -   111 replay unit    -   112 recording unit    -   121 ear authentication unit    -   122 signal generation unit    -   123 acoustic characteristic extraction unit    -   124 ear verification unit    -   125 feature storage unit    -   131 voice authentication unit    -   132 talker feature extraction unit    -   133 voice verification unit    -   134 feature storage unit    -   141 authentication integration unit    -   200 biometric authentication device    -   201 interaction control unit    -   202 simultaneous notification unit    -   500 information processing device    -   501 CPU    -   503 RAM    -   504 program    -   505 storage device    -   506 recording medium    -   507 drive device    -   508 communication interface    -   509 communication network    -   510 input/output interface    -   511 bus    -   507 drive device    -   508 communication interface    -   509 communication network    -   510 input/output interface    -   511 bus

1. A biometric authentication device, comprising: a memory configured tostore instructions; and at least one processor configured to execute theinstructions to perform: inputting a prove signal into an ear of a user;extracting an ear acoustic feature of the user from a reverberationsound of the prove signal outputted from the ear of the user; extractinga talker feature of the user from user's voice record; authenticatingthe user based on each of the ear acoustic feature and the talkerfeature; and identifying the user with a first biometric authenticationresult based on the ear acoustic feature and a second biometricauthentication result based on the talker feature, wherein the at leastone processor is configured to execute the instructions to perform:informing the user an instruction for implementing a first biometricauthentication based on the ear acoustic feature and a second biometricauthentication based on the talker feature continuously; and making useof an audible sound informing the user a timing to speak as the provesignal for the first biometric authentication based on the ear acousticfeature.
 2. The biometric authentication device according to claim 1,wherein the at least one processor is configured to execute theinstructions to perform: causing an earphone worn by the user to outputthe prove signal.
 3. The biometric authentication device according toclaim 1, wherein the at least one processor is configured to execute theinstructions to perform: recording the user's voice using a microphone.4. A biometric authentication method comprising: inputting a provesignal into an ear of a user; extracting an ear acoustic feature of theuser from a reverberation sound of the prove signal outputted from theear of the user; extracting a talker feature of the user from user'svoice record; authenticating the user based on each of the ear acousticfeature and the talker feature; and identifying the user with a firstbiometric authentication result based on the ear acoustic feature and asecond biometric authentication result based on the talker feature,wherein the biometric authentication method comprises: informing theuser an instruction for implementing a first biometric authenticationbased on the ear acoustic feature and a second biometric authenticationbased on the talker feature continuously; and making use of an audiblesound informing the user a timing to speak as the prove signal for thefirst biometric authentication based on the ear acoustic feature.
 5. Thebiometric authentication method according to claim 4, wherein thebiometric authentication method comprises: causing an earphone worn bythe user to output the prove signal; and inputting the prove signaloutputted from the earphone into an ear of the user.
 6. The biometricauthentication method according to claim 4, wherein the biometricauthentication method comprises: recording the user's voice using amicrophone.
 7. A non-transitory recording medium having a biometricauthentication program stored thereon, the biometric authenticationprogram causing a computer to perform: inputting a prove signal into anear of a user; extracting an ear acoustic feature of the user from areverberation sound of the prove signal outputted from the ear of theuser; extracting a talker feature of the user from user's voice record;authenticating the user based on each of the ear acoustic feature andthe talker feature; and identifying the user with a first biometricauthentication result based on the ear acoustic feature and a secondbiometric authentication result based on the talker feature, wherein thebiometric authentication program causes a computer to perform: informingthe user an instruction for implementing a first biometricauthentication based on the ear acoustic feature and a second biometricauthentication based on the talker feature continuously; and making useof an audible sound informing the user a timing to speak as the provesignal for the first biometric authentication based on the ear acousticfeature.
 8. The non-transitory recording medium according to claim 7,wherein the biometric authentication program causes the computer toperform: causing an earphone worn by the user to output the provesignal; and inputting the prove signal outputted from the earphone intoan ear of the user.
 9. The non-transitory recording medium according toclaim 7, wherein the biometric authentication program causes thecomputer to perform: recording the user's voice using a microphone.